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I. INTRODUCTION 

The Hellenic Air Force (HAF) is interested in optimizing 
its combat readiness, and conseguently the combat readiness 
of 1ts combat pilots. Greece has common borders with 
hostile countries and is constantly threatened with possible 
invasicn. There are continuous violations of the aerial and 
sea space of Greece Ly neighboring countries! airplanes and 
ships respectively. We (the Greeks) must offset our 
numerical inferiority with technology expressed in higher 
skilled combat pilots. It 1S imperative to increase to the 
utmost the combat readiness and effectiveness of the HAF. 
In this way we shall be a strong power and our enemies will 
never try to invade our country. Thus, we shall remain in 
our purpose, that 1s, united and uninvaded country and most 
important, because of our power we'll keep the peace. 

Today the need for combat effectiveness 1S more fressing 
than ever before. We live in a period of rapid and 
continuous change in technology and conseguently in our 
tactics. The new generation of combat aircraft reguire more 
training because of the complexity of the new systems. 
Pecause of these technological changes it is required that 
new changes in air tactics must be established. Thus, 
because of the above continuous changes, it is required that 
new criteria for comkat readiness of combat pilots must be 
established (created). Of course, HAF has established 
criteria for its comkat pilots as a result of the entry of 
the present generaticn of combat aircraft. However, it may 
be necessary to see if there are any critical points that 
need to be corrected or changed. For these reasons this 
study has been conducted in which the US Air Force was used 
aS a model and a later study will be necessary to adapt 
these models to the Greek reality. 
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Thus, this study will define combat readiness, the 
criteria and their considerations and human performance, in 


relaticn to the combat readiness of combat pilots. 


A. COMBAT READINESS 


But what 1S combat readiness? In order to answer this 
question, we first must define the word readiness, secondly 
combat readiness, and third readiness categories. 

Readiness means the guality of being ready to act, 
respond, comply, etc; Also readiness is a state or fact of 
being ready, or prepared; as to have everything in readiness 
for a sudden departure [Ref. 2]. A crew or crew member is 
combat ready when it 1s certified as ready for combat 
fRef. 3]. 


Combat readiness is expressed with the following 


standard categories and meanings [Ref. 4]. 
a. Fully Combat Ready. 


A pele ae Ly: capable of performing the 


mission(s) for which it is organized or designed. 
b. Substantially Combat Ready. 


A unit capable of performing the mission(s) for 
which it 1S organized or designed, but having minor defi- 
ciencies which could reduce its effectiveness or its ability 
to conduct sustained operations. 


c. Marginally Combat Ready. 


A unit with major deficiencies of such magnitude 
as to severely i2niteits Capability to perform the 


mission (s) for which it iS organized or designed, but 


1] 


capable of conducting limited operations for a  lamited 


period. 
d. Not Combat Ready. 


A unit not capable of perfoming the mission (s) 
for which it is organized or designed. 

Definiticns of readiness and combat readiness 
were given above. A classification of combat readiness into 
four categories was made in an effort to specialize and 
higil1¢ hte In order to define and understand comtkat 
readiness better we have to define criteria and their 
considerations, that is to say, definition and purpose of 
criteria, the categories of criteria and characteristics of 


good criteria. 


EF. CRITERIA AND CONSIDERATIONS 


1. Definition and Purpose of Criteria 


Criteria are standards, rules, or tests ky which 
measures oi system behavior are evaluated in terms of 
Success or failure, or to some degree of failure. The 
purpose of human performance criteria 41s to provide 
Standards or baselines for evaluating the success or 
failure, goodness or badness, or usefulness of human 
performance [Ref. 5] and [{Ref. 6]. 

Criteria should not only define the unique manner in 
which the operator shculd perform a task, but shouid define 
the performance objectives of the entire man-machine system 
(Ref. 7] and [Ref. 8]. 


2. Categories of Criteria 


The classification of criteria can be accomplished 
from a measurement standpoint; beginning with the smallest 
known entity and ending with the "ultimate" quantity that 


May €xist. Several categories identified are listed below: 
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( 1) Parametric referent or standard of performance 
which is sought to be met by the operator or system. 
Example: maintain 1000 feet of altitude [{Ref. 7] and 
(ekeks 7 1. 

(2) Parametric limit about the parametric standard 


within which the operator or system is required, or 


seeks, to remain. Example: Maintain plus or minus 
100 feet while at 1000 feet altitude [Ref. 7] and 
[ Ref. 9]. 

(3) System component criteria which distinguishes 


the relationship between system components and system 
output. Example: " least effort " measured from 
the pilot in relation to Maintaining altitude 
[ Ref. 10]. 

(4) Test criterion used to evaluate overall human 
eli ty., usually expressed as a single overall 
measure. Example: subjective judgement of instructor 
fOr a student as to “pass or "fall". 

(5) Ultimate criteria are multidimensional in nature 
and represent the complete desired end result of a 
system. This category of criteria 1S impossible to 
quantify due to the multidimensional nature of the 
system's purpose, and hence, is a theoretical entity 
that must be approximated. Example: Any aircraft's 
mission [ Ref. 1] and [Ref. 11]. 


It may be noted that all five categories of criteria 


€ither quantified or approximated in some manner, 


with decreasing accuracy as the ultimate criteria level is 


reached. 


Cbtaining direct measures of the ultimate criteria 


for a complex system is seldom feasible. This 1s particu- 


larly true in military systems where such criteria would be 


expressed in terms of combat effectiveness or effectiveness 


ig 


in preventing a potential aggressor from starting a conflict 
[Ref. 1]. 

Therefore, it becomes apparent that we must select 
intermmediate criteria (types (1) through (4) above) in 


evaluating skilled operator behavior. 


3. Characteristics of Good Criteria 


Using actual criteria as approximations of the ulti- 
mate criteria can be accomplished by several methods that 
will be presented in the following paragraphs . 

Although there is no certain method that will lead 
to the specification of good criteria, there are scme 
considerations that can be taken into account which are 
discussed below: 

(1) A good criterion is both reliable and relevant 
{ Ref. 1] and { Ref. 10]. 
(2) Criteria must be comprehensive in that the utility of 
the individual being evaluated is unambiguously reflected. 
(3) Criteria should possess selectivity and have ready 
applicability [Ref. 10]. 

Finally, since readiness and criteria are related to 

human performance, it is necessary to define human 


performance. 


C. HUMAN PERFORAANCE 


Much of the data used in human factors consist of 
measures of some aspect(s) of human performance. Human 
performance in the centext of systems often boils down to 
consideration of how fast people can perform their functions 
and how accurately they can perform them. How well functions 
are performed leads directly to considerations of human 
errors { Ref. 12). 
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De. THE PROBLEM 


Thus, the problem is the necessity of optimizing the 
combat readiness of HAF's combat pilots, because of the 
threatened invaSion by neighboring countries. So, we have 
to identify the necessary variables which will assist the 
pianner (commander) to be certain that his combat pilots 
will meet the desired level of combat effectiveness in terms 
of pilot capabilities. 

It can be suggested that for HAF the first requirement 
1s to continuously optimize combat readiness. To accomplish 
this cbjective it is essential that combat pilots be at an 
acceptable level of combat readiness. Determination of 
combat readiness reguires definition of concept, specifica- 
tion of criteria and a definition of human performance. 

In the chapters to follow an attempt will be made to 
define and discuss the following: (a) the principles of 
human performance and (b) the criteria and their measure- 
ment. Finally, chapters IV and V will examine combat ready 
crew performance and derivation of combat-crew performance 


measurement techniques. 
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II. PRINCIPLES OF HUMAN PERFORMANCE 


A. HUMAN PERFORMANCE REQUIREMENTS 


We can best determine performance requirements only 
after we have a good set of allocated functions. In the 
absence of function allocation we will have a dirficulty 
identifying and dealing effectively with human performance 
requirements because we do not know what functions peorle 
will ke expected to perforn. 

Once we have established a human performance requirement 
we Should develop a way of measuring it. If the requirement 
relates to accuracy, then there must be a meahingful way to 
measure errors. If the reguirenment concerns manual 
processing time, there must be a meaningful way to measure 
it. For example, time per customer contact, number of items 
produced per hour, and average keystrokes per day. 

Usually the problem is deciding what measure gives the 


best indication of performance. It has been suggested that 


with a little imagination, any human performance can be 
meaningfully measured [Ref. 13]. As a mininun, human 
performance reguirements Should include statements 
concerning errors, manual processing time, training time 
necessary to ensure the Minimum Skills, and job 
satisfaction. If we do not clearly state the requirements 


from the beginning we cannot expect human performance 
considerations to be taken seriously. And when the systen 
1s operational, peorle will be left to perform as best they 
can without adequate provision for ensuring an accertable 
level of human performance [{Ref. 14]. 

Other human perfcrmance reguirements could be associated 


with training time (e.g., total time to train clerical 
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personnel to perform the basic activities should not exceed 
three weeks). Another human performance requirement could 
relate to job satisfaction (e.g., after performing an 
activity for six months employees should respond in a posi- 
tive way to their work, as measured by a guestionnaire). 

The identificaticn of human perfomance reguirements is a 
prerequisite step for a system's development process. One 
of the steps of a system's development process is the task 
analysis. Task analysis is intended to match the work to be 
done with the kinds cf people who will do it. The process 
has four main parts: 

(1) Determination of a system structure that gives the 
designer an overall view or objective for the analysis. 

(2) Identification of tasks has essentially nine primary 
considerations asscciated Wd tellers These include 
determining existing knowledge and skills, deriving skill 
level categories, identifying outputs and inputs, deriving 
lower level activities, ensuring that activities are nmutu- 
ally exclusive and exhaustive, and hatching active 
complexities with previously determined skill levels. fhe 
process may be repeated any number of times until each 
activity is assigned a single skill level. Secondary 
considerations in the identification of tasks include 
meeting the system structure objective just discussed, as 
well as meeting a full-advantage objective. A designer 
attempts to develop tasks that will ultimately take full 
advantage of the user work force. Tilsoeels ditt 16ult to 
guantify, but during the analysis process, most deSigners 
gain a feel for what is meant by taking full advantage of 
the skills available in their user population and this 
should be reflected as the tasks are being identified. 

(3) Description of each task and organization of all tasks 
into a flowchart that will accomodate the variety of 


different transacticns a new system must accomplish. 
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(4) Identification of work modules that synthesizes the 
tasks previously identified into manageable modules of 
work. A human work module is a_ set of tasks that a user 
acccmplishes as a part or all of his or her job [Ref. 14]. 
It is a -asic unit work. Usually, one or more work modules 
are combined to form a job. 
Having systematically derived work modules assists in 
the design of interfaces, and the preparation of facilitator 
materials, Such as instructions, performance aids and 


training. 


Be. FACILITATING HUMAN PERFORMANCE 


Once good work modules are developed,many types of 
materials can be developed to help ensure an acceptable 
level of human performance. All of these materials are 
based on the task analysis results and are usually prepared 
to support the work module instructions. 

Once the designer identifies tasks and determines work 
modules, he must identify the specific skills and knowledge 
required for each work module. This amounts to stating his 
assumptions of the precise qualifications of the persen to 
perform the work module. 

As the work modules are designed, the designer should 
have a set of user characteristics in mind. The design 
Should clearly envision the person performing the work. 
When this information is written down, it is called a 
Statement of Minimum Cualifications or SMQ [Ref. 14]. 
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The SMQ is a detailed description of minimum acceptable 
qualifications in terms of skills and knowledge required to 
efficiently and effectively perform the work outlined by the 
work module. 

Once the designer has in mind the potential user and has 
designed the work modules, then he can proceed with 
developing instructions, performance aids and training. 

et is vitally important that human performance 
considerations begin at least by the time functions are 
being allocated. Having a good set of human performance 
reguirements is also critical. A systematic and detailed 
analysis of the tasks to be performed 1s also very 
important. 

If design considerations are well done, then a strong 
foundation is established for developing interfaces and 
facilitators, including Human/Computer dialog structures, 
instructions, performance aids, training materials. 

In this charter the principles of human 
performance, namely (a) Human Performance requirements 
and (b) Facilitating human performance have been discussed. 

In order to measure ore person's attitude, aptitude and 
performance we may use criteria. Thus, in the next chapter 
we Shall expand on the concept of criteria and their 


measurement. 
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III. CRITERIA AND THEIR MEASURMENT 

As suggested earlier, a criterion is a standard which 
may be used to evaluate a person's attitude, aptitude, and 
performance. Since personnel selection and training are 
important factors for HAF to optimize combat readiness of 
its ccmbat pilots, we will be concerned with the use of 
criteria for personnel selection and training purposes. 

Selection criteria are descriked by the degree of ccrre- 
lation between selection test scores and performance 
measures (in real-world situations), which represent the 
degree of a pilot's success in performing his job. 

Training criteria refer toc measures utilized in evalu- 
ating the effectiveness of a training program (i.e.,the 
measures that express the degree to which the attainment of 
the behaviorial objectives of the training program have been 
met) [Ref. 15}. 


Ae USE CF CRITERIA IN PERSONNEL SELECTION AND TRAINING 


Personnel selection serves to predict a person's 
Suitability for a ‘jcb, whereas the purpose of personnel 
training is to derive a predetermined work standard, Or 
Gunel CEiter Lon, in the shortest possible training time. 
Both purfoses can be achieved by analyzing and quantifying 
the content and skills associated with the job. The valida- 
tion of each process can ke evaluated as follows : 

1. The validity of a personnel selection system can be 
evaluated ky the degree of the relationship, or correlation, 
between test scores against the criteria measures. 

2. The validity of a training method can ke evaluated by 
comparing a variety of different training methods with a set 


of the same criteria for all cases. 
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The validity of selection and trainingj programs alike is 
affected by the follcwing factors 

1. The degree to which personnel selection tests or 
personnel training prccedures simulate real-world criteria. 

eae The validity of personnel selection tests and 
training under study. 

3. Validity of the criteria considered, true and not 
merely ccrrelational. 

YW. Relevance of the criteria considered for (a) the job 
studied, and ({b) the test batteries adopted for selection or 
training purposes. 

5. The stage (initial, ultimate, or rate of learning) at 
wnich criteria are measured may affect validities signifi- 
cantly. Typically, criteria measures at the early stages of 
the acquisition of skill exhibit higher validity coeffi- 
clients than those we find when the measures are correlated 
with terminal performance [Ref. 16]. 

6. Because job performance is multidimensional, taking 
only one criterion as an index of job performance may result 
in dettticially nigh Or low validity coefficients. 
Therefore, a multidimensional approach is essential in the 
measurement of job performance and criteria utilization in 


personnel selection and training. 


ys Evidence indicates that criterion and test measures 
are typically nonlinear [Ref. 17] [Ref. 18] and [Ref. 19]. 
Since, moreover, assuming their linearity increases’ the 


error variance of the validity coefficient, the validity of 
the assumption must ke tested before linearity is assumed. 
See tLuctldttlons ine dl “individual*s physiological and 
psychological health condition, need of achievement motiva- 
LON, and so on, affect the validity coefficient of 


personnel selection programs as well as training programs. 
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In summary, it is hardly possible to overstress the 
importance of a scund criterion measure against which 
personnel selection and training programs can be evaluated 
effectively. In order to achieve this objective, emphasis 


must ke placed at the outset on the development cf such 


measures. Without sound criteria the true validity of 
personnel selection and training cannot be assessed 
properly. 


Be TYPES CF CRITERIA. 


Criteria for the purposes of personel selection and 
training lie on a continuum scale, which has two interactive 
dimensions ; objectivity-subjectivity and employee-employer 
Satisfaction. Some criteria are entirely subjective; others 
are entirely objective. Most, however, involve a mixture of 
the two. In other words we distinguish two types of 
criteria: (1) Those describing the satisfaction of the 
empoyee's needs and (2) those describing the satisfaction of 
empoyers in their empecyees. 

The ideal criterion, although rarely found in industry, 
is the one that reflects the satisfaction of employee's 
needs as well as that of empoyers'. A criterion like this 
is used more freguently for personnel guidance purposes than 
in personnel selection and training [Ref. 20]. 

It is essential to aim at using the most appropriate 
CLItefloen fOr “aww specific joo, Tather than the most 
convenient criterion, but it is not always feasible to 
assess the former or to obtain data on it. Pnuise the 
analyst is frequently ‘faced with the practical issue of 
using the most appropriate criterion which data are 
available or can easily be collected. 

The reason for using a criterion varies since occupa- 


tional jobs are diiferent; conseguently, a variety of 
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criterion measures have been developed and used during the 
last century, each with its characteristic weaknesses and 
strengths. However, any criterion could be the most apfpro- 
priate for a specific job, depending on circumstances and 
purpose. The majority of the criteria relevant to today's 
occupational circumstances and competence are listed in 
Table 1. ® 


For scme of the criteria mentioned in Table 1, objective 


data can be gathered (e.g., length of employee's service), 
cthers require subjective assessment (e.g., iscyerien Gols 
ranking by superviscr, subordinates, members of equal 


status, and the employee himself). For some criteria (€.g., 
quality of output) both subjective and objective technigues 
can be utilized. Many of these can be related to training 
progress at the follcwing stages: (a) immediate, at the 
beginning of training; (b) intermediate, at the end of 
ieadarnang:; i (c) ultimate, in real-world operations, after 


completion of training. 


C. CRITERIA MEASURES 


There are basically three types of methods by means of 
which criteria data can be collected : 
Rating or ranking. 
weeecountang (e.g., the number of items produced, the 
number of accidents, the duration of service). 
3. Establishing work standards [Ref. 21]. 

In this chapter we have seen the criteria and their 
measurement and spécifically (a) Use of criteria in 
personnel selection and training, (b) Types of criteria and 
(c) Criteria measures. Next we Shall introduce the combat 


ready crew performance mneasurement. 
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TABLE 1 


Types of Criteria Utilized for Personnel Selection & 
Training 


ae Criteria reflecting employees’ competence 
A. gaa ee performance 
- Quality vommoutpur 
2- Spoiled work: 
a) amount 
thi aes 
3. Accidents: 
a) number. 
{3} cost (financial and human) 
reakages (tools,etc.): 
num ber 
Dy cos t 
ie stakes in operation: 
f numker 
COS 


oh Mo 


ariability in performance 
7. Rate_of advancement. 
*8. Standard trade examinations 
* Tie deeaeen ie 

(a) cos 


a 
| | 
| | 
| 
| 
| 
| | 
| 
| Hae to employer, (11) to employees | 
Je uration | 
B. Quantity of performance i 
le QUantiley Clout pic aes | 
2. Earnings on a commission basis 
3. Earned ronus 
4. Peak performance 
5. Lowest performance 
*6. Rate of advancement. ; 
*7. Standard trade examinations | 
| *8. DE eee : | 
(a) cos A 
i ee to employer, (11) to employees | 
| _. (b) duration | | 
II. Criteria reflecting employees" circumstances 
1. Length of service 
2. Labor turnover ; 
3. Lateness (categorized according to how late): 
a) numbers 
tf cost 
4. Absenteeism ; 
(a) certified . 
(1) number, (11) cost 
| (b) uncertified _ | 
| (1) number, (11) cost | 
hr pet eaa reflecting employee's satisfaction { 
rom jo 
| ale Rating of employee's eed fOr eae oe Vow 
2- Rating of employee's satisfaction with 
job content and desire for job enlargement 
Or job simplification | 
| 
| | 
| 





* Starred items may reflect quality and/or quantity of 
performance. Since their relationship to performance 
1S indirect, they are less valuable and important 
than the others. 


| 
oa | 
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IV. INTRODUCTION TO COMBAT READY CREW PERFORMANCE 
MEASUREMENT 
Recently, the evaluation of aircrew proficiency in 


Skills associated with advanced flying training was prima- 
rily based upon the subjective judgement of instructor 
pilots. In addition these evaluations are supplemented by 
the mcre or less objective scores of gunnery and bombing. In 
certain areas, Such as air combat Maneuvering and 
combat-readiness determination the evaluations are totally 
objective. 

Although, efforts have been made to measure objectively 
the behavioral skills in the operational and/or crew 
training setting in an economical way, these efforts have 
not yet produced any positive results. 

A great percentage of the HAF'ts budget 1S absorted by 
the aircrew training costs, therefore, this area has beccme 
of fiscal concern. Due to the projected increase in costs 
ef operating the newer weapons systems over the present 
generaticn of combat aircraft the HAF's interest in the 
problem has intensified in recent years. Studies to minimize 
costs have shown that this can be achieved by reducing the 
training flying hours and transfer them (when/where this is 
possible) to lower ccst devices, e.g. Simulators. 

The HAF is trying to adapt modern Systems Approach to 
Training technigues to aircrew training programs. Existent 
in the Systems Appreach to Training concept is student 
advancement on individual proficiency rather than course 
length. The traditional subjective methods of evaluation nay 
be proved insufficient for the more complex aircrew training 
objectives. 


Zo 


Aa O BUG Ss 


Necessity of improving training performance information 
ope 330 2 te Mejslile 18h F directed this study to the following 
objective based on experiments and studies of USAF 
{[Ref. 22] ; 


Goal: Systematic definition of performance measures 
appropriate tO | conmbat-crew Chawn2ne needs. 
Ferformance measures will include formal statements of 
methods of measuring flight crew performance’ used 
during and at tte end of combat-crew training, and, new 
measures meaningful to combat-crew training and useful 


as tools for training research. 


This study thus is an effort to describe usable measure- 
ment tools £OE Stil 7at on in combat-crew training 


research. 


7. Instructional System Development. 





Research studies directed toward performance 
measurement in comkat-crew training are highly relevant 
today in view of USAF policy to employ a systems approach to 
flying training problems. The model for Instructional 
System Development (cf., Dept. of Air Force, 1970) contains 


the foilcwing basic steps: 


1. Analyze system requirements. 

2. Define education or training reguirements. 
3. Develop objectives and tests. 

4. Plan, develop, and validate instruction. 


X- Conduct andtevalwatesrnstruertvon- 


In support cf instructional system development, 
measures and a measurement system are necessary to: 


(1) perform analyses of systems in their operational 
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environments, (2) establish quantitative instructional 
standards, (3) provide an index of achievement for each 
kehavioral objective, and (4) evaluate alternative 


instructional content, approaches, and training devices. 


in? > pabticular, instructional system develorfment 
requires that performance standards are identified so that 
the most efficient  arpproach 1s used to train the needed 
skills and knowledge to the desired level of performance. 
Such performance standards imply performance measurement for 
koth the determinaticn of desirable approaches to training 
and fer testing student performance. 

Nel these events mentioned above should be 
considered positive and therefore this model for 
instructional system development could also be used and 
adapted by HAF. 


on, A Measurement system for the Operational 


It will be necessary to develop measurement tools 
that would be usable in the operational environment of HAF 
under the constraints that such an environment implies. 
Within the context of this reguirement, it is essential that 
an attempt be made to establish a list of parameters to be 
sensed, and the pcint-of-view taken that the parameters 
should be derived from that information that the operational 
training personnel consider to be meaningful and 


Suenificant. 


Ss | 2 -  ee 


3. An Automated Measurement Systen. 


An ancillary objective is to develop a measurement 
system that will relieve the instructor pilot, to a maxinun 
extent, from the requirement of having to record a great 


deal of information manually on the basis that such activity 


Ley 


degrades his ability to competently instruct his student. 
This does not imply that such a measurement system 15 an 
attempt at automated evaluation. The measurement systen 
Should certainly include means for transforming and 
analyzing performance information, but ultimately evaluation 
and decision for training control is a human function. 

Thus, HAF would improve and optimize the guality of 
training of its comkEat pilots by adapting an instructional 
system development, a measurement system for the operational 


environment and finally an automated measurement systen. 


Be. MEASUREMENT BASED ON COMBAT-CREW TRAINING INFORMATION 
NEEDS 


The strategy employed by the USAF was to design a 
measurement system that could acyguire that data identified 
as meaningful by training management and instructor 
personnel. Data were gathered from sites across a_ broad 
spectrum of combat-crew training programs of USAF. 

The combat-crew training sites from which data were 
taken are listed in Table 2. The aircraft sample included 
heavy (inter- and intra-theatre cargo/transport, and 
bomber) and high-performance aircraft (one- and twe-man 
interceptor and tactical fighter). An attempt was made to 
(1) consider measurement in the context of combat-crew 
training, (2) assess measurement already included as well as 
identify potential measurement indicated by combat-crew 
training personnel, and (3) assess the constraints placed by 
the environment on feasible, usable measurement systems. 

From the six types of aircraft listed in Table 2 the HAF 
pilots fly the three cf them, that is F-4E, C-130E and A-7D. 
Thus, the USAF's attempt and data found are of great 
interest for HAF, in order to improve training performance 
information for its ccmbat-crews. 
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Noi 2 
Data Collection Sites 

Place Aircraft 

Castle AFB B= 520.8, .6.-c. Hi 
| Altus AFB C-141A 

Dyess AFB C— ser 

Davis-Monthan AFB F-4 C, DEE 

Tyndall AFB P= 106 

Luke AFB A- 7D 

*George AFB F-4E 

*Ncrton AFB C-141A 

**Nellis AFB F-45 


* Special emphasis on crew-performance measurement 
**x Special emphasis on air-combat maneuvers 
performance measurement. 


hos 
™ 
to 
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C. COMBAT-CREW PERFORMANCE MEASUREMENT 


For the six aircraft mentioned in Table 2, a ccmmon 
basis for measurement was established. This way it could be 
possible to apply a mere or less modular approach. Allowance 
must be made for special aircraft characteristics; Ons 
example, the F-106 has no flaps andthe B-52 has a guite 
complicated flap retraction routine compared to other 
aircraft. Measurement was treated for each of the following 


Maneuvers: 


(1) Takeoff and Climb 

(2) Pattern, Land and Go-Around 
(3) Instruments 

(4) Formation 


(5) Air-Air Intercept 


Zo 


(6) Air Combat Maneuvers 

(7) Air Refueling 

(8) Ground Attack 

(9) Air Drop and Air Drop Formation 


(10) Radar Navigation and Bombing. 


Prototype measurement. For each of above maneuvers, the 
data collected from the sites of Table 2 were compiled into 
the summary form shown in Figure 4.1 Since the blanks in the 
summary form of Figure 4.1 indicate needed informaticn, and 
conseguently items fcr measurement development, these forms 
were termed Prototype Measurement Since they form a model 
after which measurement could be patterned. Details and 
explanations of a prototype measurement is presented in 
Appendix A. 

Measurement Specifications. The parameters which must 
be sensed to permit measurement are not immediately evident 
from the information requirements (Prototype Measurement), 
Since the measure specifies the output of a computation, and 
the ccmputation itself must be known before the inputs to 
the computation (the farameters) can be determined. 

For example, Figure 4.1 indicates that a measure of 
centerline deviations is desired during the takeoff roll. It 
is clear that the distance between the aicraft position on 
the runway and the runway centerline is a parameter needed 
for measurement. The desired measurement might be simply the 
average difference, or, conceivably, might involve the rela- 
tionship between centerline deviation and heading (or 
lateral-G, or brake application) and thereby indicate the 
need for other parameters to execute the calculations for 
measurement. Further, the measurement calculations must be 
made (as indicated in Figure 4.1 ) from the application of 


takeoff power until rotation, implying the need for cther 
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CONDITIONS: 
Gross WEt__ _ Wing:  f/ Runway:  /_ 
Tenpe>. = Mitwoctes, © ora tela Elev, Form Pos.:  _ 
TAKEOFF ROLL: (To power until rotation) 
Power Set: _ “Centerline Dev.: Min, Max, Av. 
Reject Speed: Computed pea ol nee te Max, Av. 
Time: Dales Crs Bank: Max or bea xX 





| 
| 
| 
ROTATION: (Nose gear off until pitch att. established) ! 
Rot. Speed>_ pales: Sides ions © oe | 
Peet chsaenate 2 oe = pa 
Pines. Centerline Dev.: _ | 
Overshoot? heading: ——_——_— 4 | 
MeerlOrr: (Pos. Vert. eri) | 
Airspeed: _ ~ Pitch: Bete Pg Cpc meme | 
Vert. Vel. After: SOCes | 
GEAR-UP: pee upyuntid gear=up & locked) | 
SBear=Up Speeds Voevecdnits: © V.V."Final:  _ | 
Eelticins | Bank: _ PAGS) Se eae | 
FLAES UP: (Start up to full up) #% 
eins © B-52Z Only IAS PITCH ALT VV TRIM 
Pitch: Bank ee Siecs t x X and X 
A/S (ee INAL) —_ 1st Pos X x coy xX X 
VV { ny FINAL) _ 2nd Pos X O%6 ns x | 
Ra (INLT)_ FINAL) _ Full x x x Xx x 
CLIMB & LEVEL OFF: (Depends on Flight Pian) 
NIT FINAL 
PWR A/S MACH HDG ALT ALT PITCH IRIM 
Accelerate x X x z X x X x 
eromb A/S te 3} 

; # 2 x ng X X x X X X 
Ciimb MACH De x x x x x x xX | 
Level-off (A1t-10% VV) | 

EO GEHl1 Se )ix ne X x x x x x | 


* Also, mandatory communication & instances where A/C 
limits are exceeded. 
**F—106 has no flars 


Figure 4.1 Example of Prototype Measurement. 


ot 


parameters to indicate when the measurement interval starts 
and stops. 

Since the pilots of HAF fly at most the same aircrait! 
as USAF pilots, the data gathered from these USAF [Lases are 
of great interest for HAF. Of course, of greatest interest 
are the data of the three similar aircraft (F-4E, A-7D, and 
C-130E), but the data related to the other aircraft should 
also ke of great interest. These may be applied to cther 
aircraft that pilots cf HAT tly alee. F-1C Sirage, Or 
F<~5A/B), Or aircraft that HAF will be procured ingthewiegs. 
future. 


Mine aircraft that Ene HANS Seepmoes core and are similar 
to the USAF aircraft bases from where data were gathered 
are: F-dE, A-/), and G-jis0 be 


4 
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V. DERIVATION OF CCMBAT-CREW PERFORMANCE MEASUREMENT. 


One cf the principal products of the USAF study was the 
definiticn of performance measures appropriate to combat- 
crew training needs. That study was based on interviews with 
instructors and training management and measurement defini- 


tions were evolved in the following steps : 


(1) The varied requirements posed by the six aircraft 
and missions were consolidated into a common framework which 
permitted isolation of measurement modules (measurement 


commonality) ; 


( 2) PE ScusstOns OL mmoremdt ronda) tClalning  1nierm ation 
needs were formalized to indicate in a checklist fashion the 


measurement development needed (prototype measurement) ; 


(3) Measurement parameters and (4) measurement specifi- 
cations were produced together, but are presented separately 


for hardware and software implications, respectively ; 


(5) Measurement and analysis for crew communications 
recording were examined to provide means of examining crew 
interaction and individual contributions EOmmeOtale lan 


machine system performance. 


A. MEASUREMENT COMMONALITY 


Commcn measurement was executed to permit the design of 
one simple and practical measurement system, and eliminate 
the need for a totally uniygue measurement system for each 


aircraft. This measurement system could apply in HAF. 


So 


In the USAF program the first step was to ccmpare 
Similar training phases for each aircraft, in order to esti- 
mate the commonality measurement requirements, as it is 
shown in Table 3. In cases where X-Y2 data are required for 
measurement it was found that these data could be obtained 
by using expensive eéguipment (such as a multiple-target 
tracking radar) or with difficult-to-process recordings, 
while equivalent results could be obtained with video/fhoto 
sensors. As we can see in Table 3, almost all cases of X-Y 
data requirements can be met by using video/photo recording. 
Those cases where the X-Y data cannot be obtained with 
videoyphoto sensors are: (1) lateral drift across’ the 
runway during transition, (2) relative position of aircraft 
during intercept prior to lockon, (3) enroute cross-track 
error during airdrop, (4%) inflight ranging (out of Sigae 
during formation, and (5) space paths of multiple aircraft 
during air combat maneuvers. 

Although, phcto-sensors are widely used by the HAF, 
video is not. So, the use of video films would be recon- 
mended because video is a modern technology equipment with 
many advantages (e.g., film does not need development and 
therefore can be used immediately after flying during 
debriefing). 

Not all maneuvers were taught at all sites from 
which data was taken. For example, the operational C-130 
squadron did not explicity train transition maneuvers; 
nevertheless, competent information was obtained for 
measurement. Since all combat maneuvers were not taught at 


the comkat-crew training squadrons, they had gaps in data 


<The X-Y data are not in the form of a tabulation of X- 
and Y-values, but they are position information such as the 
relationship between tanker and refueling aircraft from the 
tanker lights. 
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| TABLE 3 


Reguirements For X-Y Data 


Rangiemeb on™ Lead 
Bearing from Lead 
AAltitude from Lead 
Actual Air Release Pt. 


Formation Range 
Monge Rate 
i Oe aed 
Ground Attack TGT. Slant Range 


Aim Point Error 
Bomb Fall Line 
Flight Path 


ae Spacing 
Dae Ca relrerng Range 
Azimouth 
Elevation 
Air Combat TGT. Range 


IGT. Range Rate 

TGT. Aspect Angle 
TGT. HDG Cross Angle 
eco 
p 


Ko OW OJON WN OW OJON WN BOO JOC NEW SOWO~JO (NE tIN = 


f= f= WI WWW WW WWW WNDRINDIDINIDNIDPOD od ot 2 ot td td Sd et 


ace Path 


* RA=Reduced Beto 
**Chtainable with Video/Photo Systen, 
rut not easily otherwise. 


ees I 5 REE Ca a er i I Soy Ds ae IR ES eccias GARMIN FE I a SO A ARES AIR A ee SS, ARS Ses SN Gp es I in a Semen SIRE ces ciel cee, 


i. +) =a 


Ehase Parameter Obtainabl 
wlth 
Video/Pho 
Iransition Ground Track RA* 
Centerline Dev RA 
Bare EL. t No 
Threshold RA 
Dist. Down RNWY RA 
Pagano RA 
meee cept TGT. AZimuth No 
(Prior To Lockon) TGT. Elevation No 
IGTI. Range No 
1Gisenange Rate No 
: TGT. Aspect Angle No 
Air Refueling Tanker Range RA 
Tanker Range Rate RA 
Centerline Displ. Yes 
Lights Up Yes ** 
Lights Down essa 
Lights Fore Yes** 
Lights Aft Yes** 
Altitude Error RA 
Ar DEop CEoooemrt ack Marr or No 
Posi tone ELOLr Yes 


me Yn) 


O PO we De a OD OD ee a (DD 
)) 


200 2.50 no he oo 0 oo  < SO no F< ony 


a a a a a ne er ee Re a ee 


Ww 
Sail 


collection. They tried to fill these gaps by cross sence mugs 


where possible, with cther aircraft training. 


The following are observations of the degree of 
commonality of training phases for the six USAF aircraft and 
HAF ' Seal reiaemeda 1So; 

(1) ir ANSE TION GLE ie. Transition is a phase of 
combat-crew training. Transition maneuverS appear in a 
common fashion, but the manner in which they are perforned 
is significantly different) £cmmecritcrcent ateera ee This 
phase has teen already adapted by HAF. 

(2) se LNSTRUM ENTS Sens tie Instruments 1S a common 
Flight phase for all aircraft {also adapted by HA) jams 
conseguently common measurement (criteria) is conceptually 
possible. 

(3) FORMATION (FORM). Formation flight is performed 
by all six aircraft as a means to optimally employ composite 
flight and provide individual-ship effectiveness. It is 
considered as a common flight phase among all aircraft, 
however, a number of types of formation exist for 
specialized missions, each appropriate only to specific 
‘ aircraft. The formation phase is widely used in HAF by high 
performance aircraft (as in F-4E, A-7D, F-1C, F-5A/B) and is 
very limited among multi-engine aircraft (C-130, DE—Gy 
Albatros). Measurement differences will occur bétween the 
categories of high ferformance aircraft and multi-engine 
heavy aircraft. 

(4) AIR-AIR INTERCEPT (AA). Alr-to-air intereepe 
and weapons delivery utilizing airborne radar is acccn- 
plished with only the F-4 and F-106 aircraft (in HAF with 
F-4E and F-1C). While the F-4, F-1C and F-106 maneuvers and 
equipment differ, the same basic measurement requirements 
are presented. Almost the same basic measurement regquire- 
ments could also apply in F-5A/B of HAF, against the event 


that air-air intercert is accomplished by ground radar. 
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(5) Eo eer Gulia AN EQ YER'S (BFM) AD race eC CME Aa 
MANEUVERS (ACM). BFM and ACM are grouped together in F-4 
training, and could be grouped also in F-5A/3 and F-1cC of 
HAF, while BFM and Formation are grouped together in A-7 
training. The BFM/ACM grouping was maintained for purfoses 
of measurement problen. 

(6) AIR REFUELING (AR). Air refueling can cccur 
with fcur of the six USAF aircraft (B-52, F-106, F-4, A-7), 
but is only considered a difficult maneuver for the £-52. 
iiewAr air refueling can occur only in F-4YE and A-7 but is 
not used since there are not any tankers available. 

(7) GROUND ATTACK (GA) AND DROP. A number of 
training phases are devoted to F-4 and A-7 (both aircraft 
are availaktle by HAF) ground attack (Day, Tactical, Night, 
with various weapons and delivery modes), but common 
measurement was judged to be appropriate. 

Air Drop (ccmbat airlift mission) maneuvers’ for 
C-7141 and C-130 training perhaps superficially resemble 
level-flight ground attack, but quite different measurements 
are posed. Ground attack and drop phases have been also 
adapted ty HAF but in this casethe measurement is different. 

(8) RADAR NAVIGATTON AND BOMBING (RNB). Navigation 
by use of radar, and subsequent delivery on target of either 
Slamance O© Cargo, cccurs with most of the aircraft of the 
sample except the F-106, and compatible mission performance 
measurement is indicated. In HAF this phase is used by all 
aircraft except these that are not equipped with tcadar 
(eee, F->A/B) . 


2. Summary 
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The examination of the USAF study shows that 
measurement (criteria), for the training phases of the 
sample of six USAF aircraft, are composed as follows : 

Iransition 


Instruments 


oy 


Formation 

Air-to-air Intercept 

Air Combat 

Air Refueling 

Ground Attack 

Air Drop 

Radar Navigation and Bombing 

‘ All of these training phases (except air refueling) 

acre widely used by the aircraft of HAF. From both studies, 
USAF's and present, it 1s concluded that not all of the 
training phases are applicable to all types of aircraft. For 
this reason, it 1s reguired to ignore the measurement 
developed in cases where a specific phase could not be 


applied. 


3. Common Measurement for Maneuvers 








Each phase of flight, tentatively considered to 
permit common measurement,waS examined for detailed measure- 
ment requirements. All needed méesurements, for each maneuver 
used in the USAF study, were provided by (1) interview notes 
with Instructors and Training Managers, (2) A1LTCcrate 
Techical Orders, Dash-one flight manuals for each aircraft, 
(3) Phase Manuals, (4) Instructor Guides, and (5) other 
specialized documents. Important information, as judging 
factors and common errors were noted for each maneuver, 
along with other important remarks that an instructor pilot, 
or training manager, would consider and thus translated into 
objective measurement. 

For example, measurement reguirements were noted for 
each aircraft during takeoff and climb-out maneuvers (Runway 
ROl1l, Rotation, Liftoff, Gear-up, Mrilaps-up, Clinb ang 
Level-off). A matrix of measurement requirements was thus 
produced, allowing ccmparison across aircraft. It was noted 
that takeoff and climt-out are essentially the same for all 
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aaicratt, with the exception of the importance of some 
WaIneivers (€.0.,ecoOtation iS Critical with the F-106), vari- 
ations in aircraft design (e.g., F-106 has no flaps, B=-52 
involves a complicated flap schedule), and climb profiles 
depend on the specific mission and clearance. 

Although there were some differences, in mneasurement 
requirements among different types of atlas Greats tg the 
Similarities proved to be a lot more. Also, a modular 
approach to measurement was suggested, that is, measurement 
could be produced by examining every maneuver in seguence 
for every type of aircraft. 

All of the abcve mentioned maneuvers are used by the 
Hellenic aircraft and thus common measurement for maneuvers 
in a modular approach could also be considered by all 


aabeeratit. 


Be. PROTOTYPE MEASUREMENT. 


After the examination of measurement commonality of 
training phases and maneuvers, the USAF study developed, in 
the form of formatted measurement outputs, examples of the 
reguired information for training. That is, if a measure of 
centerline deviation was indicated to be desirable, this 
would be noted. In this way they recorded all of the known 
information reguirements for a given phase of flight, and 
then, they formed the data found into a format to resemble 
measurement output. This output is termed here as Protctype 
Measurement. 

An example of prototype measurement is displayed in 
Figure 4.1 for takeoff and climb. Details and explanations 
of this (prototype measurement) example is presented in 
Appendix A. A blank or x in Figure 4.1 indicates one or 
more numerical entries to be determined as a result of 
measurement. For every one of the training phases similar 


measurement was develcped. 


So 


In the USAF study the prototype measurement was produced 
for the folowing  thdem geo nas co. 

(1) TRANSITION. For convenience, transition was divided 
into (a) Takeoff and) Glin > 7aeeand (b) Pattern, Land Vom 
Go-arcund. AS previously noted, measurement appropriate to 
the ccmposite of six aircraft is indicated; the F-106 and 
B-52 reguire special treatment. 

(2) INSTRUMENTS. Measurement for instrument flight is 
treated as the sum of (a) basic aircraft contwom 
performance, and (b) navigation performance with respect to 
air traffic control requirements. While required measutre- 
ment modules for each of these classes of measures can be 
specified, it is difficult to present detailed measurement 
except for specific clearances and published procedures. 

(3) FORMATION. Measurement for formation flight is 
Super-imposed onto mission performance measurement, thus 
making formation measurement difficult to isolate. In 
particular, tactical formation performance was not clearly 
identified in this study in an objective guantitative 
fashion. It may be necessary to rely heavily on instructor 
subjective measurement (which may be guite satisfactory when 
the instructor is ina position to observe performance ), 
except for measurement associated with join-up, close fornma- 
tion, and in-trail formation. 

( 4) INTERCEPT. In Order to Mbeysneciaie, intercept 
mMeaSurement was based primarily on the F-106, however, the 
intercept problem is essentially the same as the F-4. Of 
cource, a two-man crew performance is expected to be better 
in the F-4. There is a radar observer to perform the scope 
work and differences in the equipment and capability suggest 


that slightly different strategies might be employed. 
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(5) AIE COMBAT MANEUVERS. Prototype air combat maneu- 
vering measurement dealt primarily with set-ups during 
initial training and for dart? firing. 

(a) Air combat set-ups involve placing attacking and 
defending aircraft in fixed initial positions, then freeing 
them to perform a maneuver, and subseguently judging from 
the final position whether the maneuver was properly 
performed and whether proper advantage of the tactical situ- 
ation was taken. Thus, measurement can be directed to 
description of the maneuvers performed (e.g., hard turn, 
Met o-speed yo-yo, Scissors, barrel roll, etc.), and to 
determining whether a given student was able to improve his 
Situation. 

(b) For dart firing the prototype measurement assumes a 
Eutterfly pattern or the equivalent. A pass is made over the 
target, atime hack is taken crossing the dart, the pilot 
must circle back to make an intercept to put a hole within 
the target in a given amount of time. Thus, the time and 
hits on each pass is’ measured; additionally the range, 
azimuth, and elevaticn at the beginning and end of firing 
describe the firing frosition. Fouls are called for low 


airspeed and for firing within a minimum firing range. 


(6) AIR REFUELING. As Air refueling is especially diffi- 
cult in 2-52 combat-crew training, the prototype measurement 
was tailored to the E—-52 tasks and to Strategic Air Command 


requirements. 


(7) GROUND ATTACK. During training, ground attack is 
divided into ground attack, G(ECwMd attack  nagnt,; ground 
attack tactical. Ground attack measurement was dictated by 
information needed for standard error analysis of weapons 


delivery accuracy and by ground attack procedures. Some of 


a i EE ibe ees te et) ee ce ep ye 


. 3The dart is a kind of air to air target, towed by a tow 
aie Tat t.. 
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the measures are specially designed to apply to the A-/7D 
heads-up display. 

(8) AIR DROP. Extensive prototype measurement was 
dictated for the Comktat Airlift Mission since very detailed 
procedures are adhered to throughout the many portions of 
the mission. Crew performance is especially important in 
this mission. 

(9) RADAR NAVIGATION AND BOMBING. Prototype measurement. 
for Radar Navigation and Bombing is heavily dependent on the 
characteristics of the avionics used. As only the B-52 is 
eguipped with low-level terrain avoidance radar (of the 
alrcrarft sampled in this study), measurement for these 
Maneuvers has been tailored to this application. 

Of course, from the examination of the protctype 
measurement we can see that the measurement reguirements are 
very extensive and ccmplex. For example, it may be seen that 
to describe just Takeoff and Climb, it 1s necessary to 
compute 50-100 numbers. Subseguently, if full mission 
measurement is reguired, including transition, instruments, 
formation, and wearens delivery, it will require a very 
large set of descriptive numbers. But probably, this detail 
is very necessary to support the training process. The 
instructor may need considerable detail to perform his job 
well. AS a conseguence, the prototype measurement could be 
very useful for HAF, since today the evaluation of aircrew 
proficiency in skills associated With advanced flying 
training iS primarily based upon the subjective judgement of 
instructor pilots. 


C. MEASUREMENT PARABETERS. 


Measurement is the process of producing measures which 
are indices of perfcrmance such as the conditions existing 


at the time of weapon release during ground attack, 
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deviations from the clearance during IFR flight, and flight 
conditions at lift off. The measures are commonly computed 
from flight variables (e.g., altitude, airspeed, heading, 
etc.) and other raw information which must be recorded at 
some time; these are termed measurement parameters. The 
measurement parameters must be specified so that the 
required sensors and recording equipment can be determined. 
However, Since the measures are the result of a computation, 
the details of the computation must be known so that the 
inputs to the computation (the parameters) can be 
estaklished. 

Figure 5.1 depicts the relationship between the speci- 
fied measures, the computation, and the measurement parame- 
ters. The corresponding data processing is shown in Figure 
eee It is assumed that the flight maneuvers will be 
divided into segments, so that different measures nay be 
computed as appropriate for each segment (for example, 
different measures are required during takeoff roll than 
during climb- out). Consequently, it may be seen that the 
method of determining when to start and stop the computation 
of a specific measure may reguire the recording of measure- 
ment parameters in addition to those reguired during measure 
computation (e.g., the recording of weight-off-the-wheels to 
iandicate liftoff). 


43 


SUdTOWRULg JUaWaUNSPaW JO UOLZRILJLYUaP] T's sunbl4 











WLYG 
JONVWHOSY 3d SNOTLVLAdWOD 
Gau1S3d dOLS-LUVLS 
404 GauINdI’Y 
SudLaWvuvd 
Saunsv an NOILVINdWOO yI90) to ee 
dOLS-LUVLS W SudLsAWVuVd 


eSene 


44 


PARAMETER 
\V/ 


> SAMPLE 






CALCULATE: 


¢ Value 
> Min 
¢ Max 








Mean* 
Std Dev* 


ENTER VALUES 
INTO DATA 
BASE 





START 
PARAMETERS 


STOP 
PARAMETERS 


*ONLY WITH 
AUTOMATIC 
PROCESSING 


Figure 5.2 Example Raw Data Processing 
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In Figure 5.1, the output measures (0) correspond to the 
inforfation requirements symbolized by the blanks in the 
prototype measurement forms; that is, if the protctype 
measurement forms indicated that a measure of centerline 
deviation is needed, then parameters must be recorded and 
computations developed which will answer the required 
measure, the following types of parameters may be needed in 
addition to the basic test parameters (M) (for example just 
mentioned, the basic test parameter would be a recording of 
the deviation from the runway centerline during takeoff 
roll): (1) parameters for implementing logic to start and 
stop measurement computations (S), (2) information related 
to desired performance (D),and (3) error information derived 
from the differences ftetween actual and desired performance 
(te) In short, given output measures (0), to determine 
other parameters which must be sensed (M,S,D,E), it is 
necessary to determine logic and computations to be used in 
measurement data Frocessing (1.-€e., the measurement 
algorithms). 

Assuling automated measurement, i.e., parameters are 
automatically recorded for subsequent computer analysis, the 
primary details of measurement computation (Figure 5.2 shows 
a representative flow diagram) are presented in Figure 5.3 
for eé€ach maneuver and maneuver segment of combat-crew 
training phases. The figure 5.3 indicates the name of each 
measure, the specific function to be computed, and _ the 
Starty/ystop conditions for controlling computation. FOL 
example, centerline deviation during the takeoff roll is 
desired output information, the average, minimum and maximun 
deviation are the specific computations which should be 
performed between brake release and rotation. Comments are 
also provided as considered appropriate by the analyst to 
point up alternatives, or where problems may be encountered 


during design. 
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Since figure 5.3 indicates the functions to be ComEuted, 
the conditions under which computation should occur, and 
indirectly the source information upon which to base compu- 
tation, the basic information is provided to allow prepara- 
tion of computer programs for automatic measurement. 

When a list of parameters was developed to show all the 
required sensors and recording needed for total measurement, 
it was seen that the resulting parameters could be placed in 
the following overlapring categories: (1) pictorial inft@re 
mation (e.g., out- the-window radar), (2) analog information 
{eo gay time-varying quantitative, such as airspeed), 
(3) discrete infcrmation (e.g., weapon release), (4) audio 
information (e.g. , communications), and (5) desired 
performance and existing conditions. These parameters were 
later assigned to alternative devices for data acguisition. 
After tradeoff analyses were conducted, a hybrid audio- 
videoyfhoto -digital recording system was adopted. Although 
many parameters could be acquired by either video-fheto or 
digital recording devices (to allow a partial system to have 
a stand-alone capability), tentative parameter allocations 
are listed in Tables 4 and 5. Audio recording will be 
accomplished with either video-photo oor digital recording 
devices. Desired performance and conditions are manually 
derived from briefing/debriefing sessions and documents. 
Additionally, spatial coordinates (X-Y data) may be obtained 
if data are collected onan instrumented range including 
tracking-radar equipment, although equivalent information 
may be available from video-photo recording as shown in 
Table 3. 

After detailed trade-off analyses, the use of video or 
photo techniques was emphasized due to lower costs, flexi- 
bility of application and simpler development compared to 
other all-electronic techniques; however, the problems asso- 


ciated with cockpit installation and an unfortunate tendency 
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for loss of data should be noted, but it is believed neces- 
sary to accept these deficiencies for low-cost combat-crew 
training application. 

Accuracy reguirements for measurement parameter Sensing 
and recording are listed at the right-hand side of Table 4. 
These accuracies are referenced to the information displayed 
to the crew (for example, the required airspeed tolerance is 
plus or minus 1% of the deviation between recorded values 
and those displayed to the pilot on his airspeed indicator) 
Since the criterion given the crew is that they maintain 
vehicle parameters within specified tolerances referenced to 
their instruments (typically 5-10") snocs for airspeed 
control). The tolerances listed in Table 4 are ap proxi- 
mately 1/10th the tolerances reguired of the crew. 

As a result, (a) if HAF has to make a choice between 
video/yphoto and digital recording approaches to measurement 
system design, then the video/photo recording would be 
chosen for cost, information provided, flexibility and ease 
of use. (b) A hybrid system combining the advantages of 
both is preferable to either type of recording alone. Due 
primarily to cost, the bulk of measurement parameters would 
ke derived from a video/photo system, and the remainder with 


a small digital recording capability. 


De. MEASUREMENT DESCRIPTIONS 


The gross operaticns involved in measurement computation 
are presented in the flow diagram in Figure 5.2 Each 
parameter must be sampled {at a sampling rate of 2, 10 or 20 
times a second, depending on the application) and tested to 
determine if conditions are appropriate to start measurement 
computation, and later to stop computaton. During the 
measurement interval (or at specific conditions, Ee Jey 


flaps-up), one of the following statistics is calculated: 
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TABLE 5 
Parameters for Video/Photo Data Acquisition 


Transition Runway Centerline Deviation, Lateral 
Drift, Threshold Crossing, Distance Down 
Runway, Ground Track. 
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(1) the value of a parameter, (2) minimum value, (3) maxinuo 
value, (4) mean, or (5) standard deviation. Thus, measure- 
ment is defined by specifying start/stop conditions and one 
of the five statistics. 

An example of a measurement specification patterned in 
this fashion is shown in Figure 5.3. Measurement specifica- 
tions were produced for each common training phase. 

While these specifications initially assume automatic 
recording and computation, they can also be used to describe 
Manual data processing (e.g., a discrete signal indicating 
wheels-up). These parameters can also be used to start and 
stop manual processing such as scanning for out-of-tolerance 
conditicns. During the measurement interval, the value, 
minimum or maximum cf a parameter may be determined manu- 
ally, but manual processing for computation of a mean or 
standard deviation is judged to be excessively laborious and 
time-consuming since a large number of data samples (at 2 or 
more times a second) is necessary. Consequently, the 
measurement specifications, as exemplified in Figure 5.3, 
are suitable for (1) defining software for digital comfuter 
measurement processing, or (2) manual processing procedures. 

Today, HAF specifies its measurement specifications 
mainly by manual processing procedures. Lately, HAF has been 
interesté€d in computer measurement processing. Thus, HAF 
Should ccnsider the measurement specifications mentioned in 


this section. 


E. CCMMUNICATION ANALYSIS 


Measurement for crew performance overlays the systen 
performance measurement thus far discussed. The performance 
demonstrated in A-7 or F-106 aircraft clearly involves only 
one man; however, the same mission may be flown by two men 
in the F-4 aircraft, requiring additional measurement to 
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investigate crew interaction and diagnose individual 
performance contributions. The combat airlift missicn in 
the C-130 and C-141 aircraft involves such close coordina- 
tion of pilots, navigators, loadmasters and engineers that 
it is difficult to isclate an individual's performance (even 
the crewmembers themselves cannot be sure of the adequacy of 
their performance). 

An individual's performance can be assessed by relating 
measurement at his workstation to overall systen 
performance; this type of measurement is subsumed under the 
previous measurement discussion. However, the interaction 
between crewmembers reguires analysis of communications--- 
presenting somewhat different ‘reguirements than system 
performance measurement. The following paragraphs present 


important topics related to crew performance measurement. 


1. Communication Measurement Categories. 


Communications measurement must treat at least two 
gross types of crew interaction: (1) ProOrnati on gas 
exchanged to aid another crewmember in performing his duties 
(e.g-, when the F-4 Weapon System Officer acts as a good 
"“Copilot"), and (2) a crewmember provides a directive role 
in guiding another's performance (e.g., when the F-4 Weapon 
System Officer provides directive commentary to the Aircraft 
Commander in air-air intercept). In the later case direct 
links between auditory commands and system performance can 
be identified, allowing communication to be measured in 
terms of resulting performance changes. 

Six categories of measurement related to information 


transfer were examined: 


(jee 2 Maid Measures of information timing should 
relate to (a) jammimg more important messages, (b) providing 


information at the wrong time, (Cc) delay in providing 
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information, “and cay providing information at a rate not 
permitting effective response by another crewmember. 

(2) Accuracy. Measures of accuracy reguire 
compariscn of what is said in relation to the measured situ- 
ation (e.g., was altitute reported correctly?). 

(3) Brevity. As radio and interphone traffic often 
exceed channel capacity in combat, measurement should 
address communication duration and comparison brevity code. 

(4) Number and Frequency. Also in relation to 
communication brevity, the number of communications and 
freguency cf communications can be measured. 

(5) Information content. Measures of information 
transmitted per unit time are quite important although 
uSually practically difficult to obtain. Other measures such 
as time, number and frequency are often confounded Since a 
crewmember may convey much information in a short time or 
few transmissions, while another may say little in along 
time or many transmissions; without knowledge of the infor- 
mation content it wceuld be difficult to evaluate these 
Situations. 

(6) Performance changes. The performance changes of 
the vehicle, desired as a result of communicating, define 
measurement in terms of links between auditory data and 
system/mission performance data. For example, turning 
performance can be measured following a "hard-as-possible" 


direction to the pilct of an F-4 aircraft. 
2. Auditory Data Processing. 


A computer-assisted manual auditory processing 
System is required since automatic voice decoding equipment 
is not available. Expert personnel are therefore required to 
identify complex perfcrmance and to structure processing 
Tules for data clerks who reduce data toa form allewing 


input into a computer. 
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Three auditcry data processing problems imposed 


requirements for measurement system design: 
deine tT Tear. On. 


In spite of normal communications clutter, it is necessary 
to clearly identify who is talking, even if two transnis- 
sions are Simultaneously made (jamming). Voice key 
circuitry is recommended to provide a digital signal indi- 


cating which crewmenber is talking. 
Eee ne akon LZat On. 


Audio data must be synchronized with other data 
recording to permit relating auditory information tc corre- 
sponding performance changes. Each recording device must 


therefore include an audio voice track. 
c. Data Reduction. 


Manual functions in audio data processing are 
unavoidakle, but can be minimized Chin Olulgih computer- 
assistance. Audio data playback must be accompanied by 
display cf performance parameters, especially the parameter 
of TIME, to allow ccmputer correlation of manual audio-data 
entry with the digital data base. A convenient means for 
Manual data entry is a desirable feature. 

These data processing problems must be 
considered in the design of a combat-crew training measure- 
ment system. Thus, Since HAF has at most the same aircraft 
as the aircraft of the USAF bases from which data were 
gathered, HAF must arply the same concepts for the purpose 
of deriving combat crew performance measurement. This 
approach will solve its problem of continuously optimizing 
the comkat readiness of its combat pilots. These concepts 
(measurements) that were discussed in this chapter are the 


following: 


BS 


(1) Measurement Comronality 
(2) Common Measurement 

(3) Prototype Measurement 
(4) Measurement Parameters 
(5) Measurement DeScriptions 
(6) Communication Analysis 

Of course, the value of experience in combat is 
not easily determined. History says, experienced fighter 
pilots have done better than inexperienced pilots in comkat 
Situations. Intuitive judgement says that the greater the 
experience level, the greater the chance aunit has for 
Success. 

Modern fighters are only as effective as the 
pilots who fly them. The HAF should continuously evaluate 
the need for experienced fighter pilots in the combat ready 
units. The price may ke high, but the price of defeat in the 
next battle for the air may be even greater. At the end of 
WWI (World War I) a dcctrine was written, "... aif you hold 
the air you cannot be beaten, if you lose the air you cannot 
win." [Ref. 28]. 

So far, in order to establish criteria for 
combat readiness of HAF's combat pilots, we have dealt in 
Chapter II with the principles of human performance, in 
chapter III with the criteria generally and their measure- 
ment, in chapter IV with the introduction of the combat- 
ready crew performance measurement, and finally in chapter V 
we derived the combat-crew performance measurement 
techniques. 
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VI. CONCLUSIONS AND RECOMMENDATIONS 

The Hellenic Air Force has always been interested in 
keeping the highest level of performance for its comtkat 
pilots and keep up with the latest of technology achieve- 
ments. Studies, as that of USAF, have always been of a great 
concern, for the purpose that new technigues and theories 
can be revealed in its benefit. In this way, HAF keefs up 
with the latest and increases the efficiency and effective- 
ness of its combat pilots. The necessity of improving 
training performance information of its pilots reguires new 
methods cf measuring flight crew performance during comkat 
crew training. As we have seen in this study, criteria for 
combat readiness of fighter pilots are; 

(1) Farametric referent or standard of performance. 

(2) Parametric limit about the parametric standard. 

(3) System component criteria. 

(4) Test criteria. 

() Multidimensional in nature and represent the 
complete desired end result of a systen. 

From the examination of this and the USAF study we can 
cohlude that: 

(1) Opening a new flow of information can have a major 
influence on training technology. The execution of effective 
contemporary training, the development of new training 


devices, andthe exploitation of powerful concepts such as 


adaptive training, and learner-centered instruction, all 
depend on information available through performance 
measurement. 


(2) The role of performance measurement may be that of 
causal input, a catalyst, or a weak link in a system chain, 
but the net effect of better performance measurement in any 


case iS a positive and possible revolutionary improvement. 
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( 3) The major difficulty that stands in the way of 
improvements through performance measurement is t ha 
adequate performance measurement systems do not exist where 
needed. The present study has defined a guide for measure- 
ment in a manner permitting adaptation to specific needs and 
btudgets. 

(4) The status of combat-crew measurement and other 
areas cf complex man-machine performance is reflected by the 
measurement analysis. The measurement described here 
reflects the critical dimensions of performance, and agrees 
with the measurement structure used by operational training 
personnel. Given this capability a number of future 
advances are possible. First, use of this measurement will 
lead to efficient interim and immediately available measure- 
MentelLor trainangs Second, improvements in the generation 
of optimally efficient measurement sets may be expected. 
Third, clarification of the relationship between objective 
and sub-ective measurement should be possible. None of 
these results is going to occur without an acceptable 
performance measurement System and appropriate 
experimentation. 

This section is ccencluded with the following recommenda- 
tions for future development opportunities of the HAF: 

(1) To consider measurement in the context of combat- 
crew training. 

(2) To assess measurement, that is to identify potential 
measurement indicated by combat-crew training personnel. 

(3) To assess the constraints placed by the environment 
on feasikle, usable treasurement systems. 

( 4) To establish the model of instructional system 
development. 

(5) To establish a measurement system for the opera- 
tional environment. 
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To establish an automated measurement system that would 
relieve the instructor pilot, to a maximum extend, from the 
requirements of having to record a great deal of information 
Manually on the basis that such activity degrades his 
ability to competently instruct his student. 

(6) For the purpose of deriving combat-crew 
performance measurerent to consider (a) measurement 
commonality, (b) prototype measurement, (Cc) measurement 
parameters, ({d) measurement specifications, (e) ccmmunica- 
tion analysis. 

(7) I£ HAF has te make a choice between video/photo and 
digital recording affroaches to measurement system design, 
then the video/photo recording could be chosen, as a 
performance measurement system, because of its low cost, 
information provided, flexibility and ease of use. A hybrid 
system combining the advantages of both is preferable to 
either type of recording alone. Due primarily to cost, the 
bulk of measurement parameters would be derived from a 
video/photo system, and the remainder with a small digital 
recording capability. 
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APPENDIX A 


PRCTOTYPE MEASUREMENT 


In this Appendix an example of prototype measurement is 
presented for takeoff and climb. The format used is to 
present a discussion together with prototype measurement, 
indicating through the Figure A.1 the types of information 
which are considered important to a description of pilot 
performance. 

DAR EOE © C iia 

All aircraft takeoff and climb to a cruising altitude 
and configuration. Fixed-wing aircraft perform these maneu- 
vers in kasically the same way; however, at a detailed level 
there are distinct differences between aircraft. Thus, 
measurement must be tailored to each aircraft, but the 
general structure of such meaSurement may be defined so that 
the essential elements are constant across aircraft. fhe 
following sequence is rather basic: Takeoff roll, Rctation, 
Liftoff, Gear-up, Flarps-up, Climb and Level-off. The infor- 
mation desired within each of these flight maneuvers may 


also be expressed in a substantially common manner. 


Cong terons. To properly interpret measurements made 
during a particular flight, information on the conditions 
existing at the time are needed. The gross weight, wind 
direction and velocity, runway direction and length, 
temperature, altimeter setting, field elevation, and posi- 
tion of the aircraft in formation, are reference data for 


the evaluation of performance. 


Takeoff roll. The takeoff will be assumed to begin with 
the application of power. The takeoff roll maneuver will be 


considered finished at rotation. The objective is to 
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Figure A.1 Example of Prototype Measurement. 
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accelerate ina straight line along the centerline, Or 
parallel to the centerline, with wings level. Power and 
resultant acceleraticn must be checked; for heavy aircraft 
and/for short field takeoffs, accelaration checks are 
formally performed. Time and distance along the runway are 
checked against airsfeed to determine if necessary accelera- 
tion performance is lacking in the time to safely stop the 
aircraft. Reject speed is noted in case of an emergence. The 
formation flight leader must slightly reduce power to allow 
a margin of thrust control for other members of the flight. 


Rotation. Proper rotation is normally necessary to 





achieve predicted takeoff perfcrmance. Rotation will be 
defined as the activities between the time that the nose 
gear lifts off the runway until the time that a stable fitch 
attitude is established. Stabilizer trim is important, bank 
angle, centerline and heading deviations should be small. 
Rotation should occur within 1-2 KIAS of the desired rota- 
tion speed. The rate of rotation should not be either too 
large or too small. A specific pitch attitude should be 


established witout overshoot or oscillation. 


id Tee t i Liftoff is a dicrete event, occurring when 
vertical velocity is fositive. At this time, the airsfeed, 
pitch angle, bank angle, and heading are noteworthy. The 
vertical velocity a short time after liftoff may also be 
measured to indicate whether the aircraft is positively 
airborne, or if there is any tendency to settle back tec the 


runway. 


Géar-up. Measurement should be taken from the time that 
the gear handle is raised until the time that the landing 
gear are up and locked. The initial speed at which the gear 
are raised, the change in vertical velocity during the time 
that the gear are coming up, and pitch, bank, and heading, 


Should be measured. 
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Filaps-up. Flaps-up measurement is treated in somewhat 
the same manner as for gear-up, for the tasks are somewhat 
the same: a configuration change is occurring which 
Presents a perturbation in longitudinal control. A trin 
change cccurs, and pitch, bank, and heading must be 
Gontreclied: Normally, flaps must not be raised before a 
specific altitude and airspeed (but before maximum flaps 
speed), and during the transition to flaps-up, changes in 
airspeed, vertical velocity, and altitude indicate whether 
the maneuver is properly performed. 

The B-52 presents a special measurement reguirement 
Since a specific speed schedule must be maintained as flaps 
are raised; in addition to airspeed, pitch angle, altitude, 
vertical velocity, and stabilizer trim are of interest 


during this period of time. 


Climb and Level-off. For each aircraft, there area 
number cf methods for climb-out depending on the flight 
plan, and desires fcr economy or performance. It may be 
desirable to measure climb performance from liftoff, or to 
start when the aircraft is in a clean configuration. This 
phase may be divided into the following parts: acceleration, 
Maintain climb airsfeed (may be several increases in 
airspeed during the climb), maintain climb Mach number, and 
level-off (normally level-off begins at an altitude which is 
kelow cruise altitude by 10% of the vertical velocity). 
Power, airspeed, Mach, heading, initial and final altitude, 
pitch angle, and trim, are parameters which may be measured 


during each portion of climbout. 
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